Repository navigation
Answer 500 when a household job's commit is unconfirmed, and move sim's database code to the SQLAlchemy ORM - #1090
Merged
Conversation
When the commit fails after the job's row was inserted, PostgreSQL may already have applied it. The household job store is to raise EconomyRecordOutcomeUnknownError then, the service HouseholdCalculationOutcomeUnknownError with the job's ID and the commit error's type, and the route a JSON 500 internal_error, logged as household_job_outcome_unknown. Guard tests: a failed insert statement and a refused reference keep their own errors, and a failed insert still answers database_unavailable and logs household_job_not_created. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the database fails after the household job's row was inserted, PostgreSQL may already have applied the commit, so the job may exist. The household job store now tells a failed commit from a failed insert and raises the existing EconomyRecordOutcomeUnknownError, as the economy and household report creates do. The service starts no call and raises HouseholdCalculationOutcomeUnknownError, and POST /household-calculations answers the JSON 500 internal_error, which the Public API reads as a job that may exist, and logs household_job_outcome_unknown with the commit error's type. 503 database_unavailable now means no job was written. economy_run_store imports household_job_store, so create_job imports the error inside the method rather than at module level. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e emits Review round 1's quality findings. The household job store's tests take their commit-failing and statement-failing transactions from a new support module instead of adding another copy. The insert-failure service test now checks only that the service logs household_job_not_created, the event the unconfirmed-commit test checks is absent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The proof fixtures for the new rule sim.persistence.orm-only: the bad one is the Core household job records the persistence library has today, shortened; the good one is the same records on a mapped class and a Session. The rule's tests fail until its check, orm_access_support.py, exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add sim.persistence.orm-only: code that reads or writes database rows declares tables only as ORM mapped classes and runs every statement through an ORM Session against them. orm_access_support.py checks the source (Core tables, column collections, engines and connections outside a library's engine.py, statements not on a mapped class, raw SQL and sqlmodel), and test_every_table_is_mapped_by_a_class imports both persistence libraries and reports each table no mapped class maps. The rule starts ratcheted: rules/baselines/orm-only.txt lists the 818 findings in today's Core code, the test fails on any other finding and on an entry that no longer matches one, and issue #1097 tracks the conversion. sim.stage12.persistence-typed-dml-only no longer says "Core", which the new rule contradicts; its checks are unchanged. The unused policyengine_fastapi.database module, the only sqlmodel import, is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 2's fixes. The statement check accepted only a mapped class or a SQLAlchemy call as a statement's first argument, so it rejected ORM statements on labels, exists(), variables, select(*columns) and subquery columns; and Core reached through table(), a MetaData's tables, a mapper's local_table or session.get_bind() passed. The check now flags what reaches a Core table instead: a statement or select_from/join with a Core table as an argument (a name bound to Table() or table(), or imported under another name), .c or .columns of one, __table__, local_table, persist_selectable, sorted_tables and a MetaData's tables, table() and column(), and get_bind(). Mapped classes no longer need recognising in source, which removes the disagreement with the runtime check over DeclarativeBaseNoMeta; the runtime check accepts both bases. Unit tests cover each check, and the baseline is regenerated: 811 entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 3's fixes. One repository-wide set of table names made ordinary names such as regions and pyarrow's Table count as Core tables in unrelated modules, while a table's own methods, Core aliases, module attributes and session.query() went unreported, and engine.begin() opened a Connection with no finding. The check now resolves Core tables within each module, through its own imports (relative ones and module attributes included), SQLAlchemy Table annotations and names assigned from a table, and reports every reference to one, which covers statements, methods, column collections and joins in one check. begin() on an engine is reported, and so is importing Connection even in a library's engine.py. Baseline entries now name the table referenced instead of whole statements; regenerated, 876 entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 4's fixes. The check now resolves a dotted module path (import a.b without as, a package's re-exports, star imports), counts names and attributes assigned from a module's table (jobs = tables.household_jobs, self.jobs = household_jobs), and reports tables and selectable of a mapper however it is reached (inspect(), class_mapper(), object_mapper(), .mapper or a name bound to one), a mapped attribute's table, and the TableClause, ColumnClause and TextClause constructors. Two files with one module name are refused instead of one replacing the other. begin() counts as opening a Connection only on a name that is an engine, not on any name containing "engine". Unit tests cover each route, and the runtime check's DeclarativeBaseNoMeta path; the baseline is unchanged at 876 entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 5's fixes. A class attribute declared as a table, such as a dataclass field table: Table, now counts as one wherever it is read, in its module and in modules that import from it, so shared_rows' index.table, economy_rerun_records' key.table and the Stage 12 report run store's result_rows.table are reported. A tuple assignment pairs each name with its value. A mapper held in a name assigned __mapper__ or .mapper is followed. Annotating self.x as a Table no longer makes the bare name x one. The rules that go by a method or attribute name alone (.table, .connect(), .cursor(), .begin() on an engine) apply only in modules that import sqlalchemy. Unit tests cover each route; the baseline is regenerated: 880 entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 6's fixes. Matching a class attribute declared as a table in every module that imported from its module went by the attribute's name alone, so it reported unrelated reads (economy_rerun_records' key.table is a column) and, because one module's exports fed another's, could miss an export or never settle. A class attribute now counts only within the module that declares it; another module's read isn't followed, and the reference that stored the table there is reported where it was made. Once a library's declarations are gone, every route to a Core table starts at a reported reference such as __table__, so the end state needs no cross-module tracking. The docs list the eight rules that apply only in modules importing sqlalchemy, the cross-module limit, and how the schema comparison inspects the database through inspect(engine) in engine.py. Baseline: 877 entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 7's fix. A mapped attribute's .table was reported only on a chain rooted at a capitalised name, as in HouseholdJob.id.table, so a column held in a name or parameter (key.table, job_id.table) reached its Core table unreported, contradicting the docs' claim that every route to a Core table starts at a reported reference. Every .table read in a module that imports sqlalchemy is now reported, except a .table that is called, such as the sa.table(...) constructor. This brings back the Stage 12 report run store's two result_rows.table reads and economy_rerun_records' key.table, which is a column's table. Nested tuple assignments are paired too. The docs say how the schema comparison reads the models' columns and keys without a finding, and that instance attributes, like class attributes, aren't followed into other modules. Unit tests cover .table through names and parameters, nested tuples and an import cycle. Baseline: 880 entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 8's fix. Every .table read is reported, so a converted field that still holds its model under the name table, such as ResultRows.table, would stay a finding. The docs and the rule test's failure message now say to name such an attribute after what it holds, such as model. The docs also say which reads of another module's attributes are reported, that a called .table isn't a read, and that the end state reports the ordinary routes to a Core table, with the listed limits as the exceptions. Unit tests cover a called and an assigned .table, and both modules of the import cycle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 9 found no bugs; these are its quality findings. The naming guidance and the failure message say that .table reads are reported in modules that import sqlalchemy; the limits say tables and engines without a Table or Engine annotation, and no longer claim mapper routes the check doesn't follow. A CheckConstraint's condition may be SQL text, as a server default and an index condition may. The import-cycle test now needs a second pass over the modules to pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The conversion tasks moved ahead of the cliff work and are now C1 · 33a to C1 · 33e. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They cover what C1 · 33a builds: a typed mapped class for every table in both persistence libraries, whose Core table stays under the name the stores import; a session factory and a session with a per-transaction statement timeout; and the orm-only check counting a name assigned from a mapped class's __table__ as a Core table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A name such as household_jobs = HouseholdJob.__table__ stopped counting as a Core table, so once the tables become mapped classes every store's references would have left the orm-only findings without being converted. __table__ and the other attributes that reach a table now make the name a Core table, in its module and through imports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both persistence libraries now declare their tables as SQLAlchemy 2.0 mapped classes, with one declarative base per MetaData in each library's orm_base.py. Every column, type, default, constraint and index keeps its name and definition. The read-only models gain each table's real primary key, which a mapped class needs, so parameter_values now names its id. The stores are unchanged: each table module keeps the old table names as aliases of the classes' __table__, and those references stay in the orm-only baseline until the stores are converted. The baseline falls from 880 to 612 entries. The persistence library also gains create_session_factory and bounded_session, the session counterpart of bounded_transaction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Modal worker role's privilege list gives it only INSERT on the result tables, but a session insert of a result class also reads created_at back, which needs SELECT. Also cover the flush at a bounded session's commit, which runs under the session's statement timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Modal worker role holds only INSERT on the result tables, so a session insert of a result class no longer reads the row's server defaults back. Also name, in the orm-only check's documented limits, the routes to a table whose later uses it doesn't follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
API v1 grants the Modal worker role SELECT and INSERT on the result tables, so a session insert can read a result's server defaults back. The previous commit stopped doing that, which left created_at unreadable once the session closed. Replace its tests with this one, and check the grant it relies on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Restore the default insert behaviour for result classes, so a written result's created_at stays readable: API v1 grants the Modal worker role SELECT as well as INSERT on those tables. The orm-only check's documentation now says which names it follows and lists the forms it doesn't. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A session insert of a result class reads the row back, so the Modal worker role's list must name SELECT as well as INSERT. The comparison tables' server defaults must reach the database rather than an explicit NULL. Also pin that an attribute set to a SQL expression has to be refreshed before its session closes, and insert a result through its class as the real worker role. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Modal worker role's privilege list now names SELECT as well as INSERT on the result tables, which a session's insert needs to read a row's server defaults back. The comparison classes mark the columns API v1's tables fill by default, so a session leaves them to the database. The session factory's documentation states that an attribute set to a SQL expression must be refreshed before the session closes, and the orm-only check's documented limits now name unannotated connections and boolean expressions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Stage 12 run, budget-window and comparison stores should take a session factory, run every statement on a Session, and retry only the opening of a session. New tests run each store on real sessions as the Modal worker role and the runtime role against a disposable database. The existing tests move to the same interface. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The household job and report stores, and the record functions they call, are to run on ORM sessions. The existing tests now pass each function a session in a savepoint of their connection. New tests end the session's connection during the flush and at the commit, hold a lock past the statement limit, and check the statements name mapped classes. The baseline already leaves out the five modules being converted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The household job and report stores open a bounded ORM session per operation, from a session factory the Entrypoint's database object holds. Their record functions name mapped classes: a new job and a forced re-run's simulation run are added and flushed at once, so an INSERT's failure is never mistaken for a lost commit, and each conditional update stays one statement that leaves the session's objects alone. The economy helpers these call accept a session as well as a connection until they are converted themselves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Stage 12 comparison, run and budget-window stores now take a session factory and run every statement on a Session against mapped classes. Opening a session sends `SELECT true`, which checks out its connection, and a session opener wraps only that step, so retrying a refused connection can never repeat a write: the statements and COMMIT run after the opener returns. The statements themselves are unchanged. Updates tell the session not to look for objects to synchronize, and bulk inserts keep sending NULL for a column set to None. The deployment check reads the live schema through the engine module and the models' mappers. The converted code's 271 entries leave the orm-only baseline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Annotations such as Optional["Table"] are now reported, apart from Literal values. A declarative base module may annotate only a class's bare __table__, not a parameter or variable of that name. The proof fixture again shows a Core table access being rejected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parsing quoted names inside annotations misread Literal values and Annotated metadata, so only a wholly quoted annotation is parsed, and the rest are a documented limit. A class's guarded __table__ annotation stays allowed in a declarative base module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The command should give its check a session factory with a one-connection pool, replace that connection if it is lost, and close it when it finishes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An inspector on an engine takes a connection for every schema read. The check ran on an engine without a pool, so one run opened 14 connections through a session pooler with a fixed client limit. The deployment command now gives it a session factory with a one-connection pool, so the role check, schema reads and rolled-back writes take turns on one connection. A lost connection is replaced, and the command closes the connection when it finishes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The command should end itself when the check doesn't finish in time, and the remaining tests pin the engine's URL, its replacement of a connection lost while idle, and its report of a connection another step kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check keeps one connection between steps. If that connection goes silent, checking it before reuse blocks with no error for the retries to handle. The command now fails the deployment after five minutes, ending the process directly because any cleanup could block on the same connection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The limit must leave room for every retry when a host name has several addresses, and must end the process even if its message can't be written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The driver gives each address of a host name its own connect timeout, so five minutes could end a check that was still recovering. Allow 15 minutes, and end the process even if the limit's message can't be written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The deployment check should repeat a step when SQLAlchemy reports the connection was lost, whatever the error's class, and the time limit should end the process even if standard error is full. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pooler ending a connection can report it as an internal error, so the check now repeats a step whenever SQLAlchemy finds the connection gone. The time limit writes its message on its own thread, so a full standard error stream can't keep the process alive, and the single-connection engine has its own connect timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Enforce the ORM rule across sim's database code
C1 · 33e removed the temporary module-level table names, but this test still required each one to be its mapped class's table, so the persistence library's test job failed and the other test jobs were cancelled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
marked this pull request as ready for review
November 18, 2026 21:55
anth-volk
added a commit
that referenced
this pull request
Nov 18, 2026
…d IDs branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
added a commit
that referenced
this pull request
Nov 18, 2026
…g main #1090 added `household_job_outcome_unknown`, which built `policyengine_attributes` by hand; the rule this branch adds requires `log_extra`. The emitted fields don't change. Its docstring now gives the current reason it names no job. The text this branch added says API v1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
added a commit
that referenced
this pull request
Nov 18, 2026
With #1090 merged, 31 of the Entrypoint's 34 log calls that pass fields were affected, and only the calls about a row name one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
added a commit
that referenced
this pull request
Nov 18, 2026
…RM conversion) into the Stage 12 log fields branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1089
Fixes #1097
Fixes #1100
Fixes #1102
Fixes #1103
Fixes #1105
Fixes #1108
Summary
This PR makes two changes:
POST /household-calculationsanswers 500internal_error, not 503database_unavailable, when the database fails after a household job's row was inserted and the commit may have been applied.sim.persistence.orm-onlyenforces that with no baseline.The conversion was built and reviewed as #1101, #1104, #1107, #1106 and #1109, which merged into this branch. This PR takes all of it to
main.Household job create
If the commit fails after the insert ran, PostgreSQL may already have applied it, so the job may exist.
HouseholdJobReferenceError. A database error once the insert has succeeded raisesEconomyRecordOutcomeUnknownError, caused by the commit's error.store_transactions.create_in_transactionnow holds this logic for the household job, household report and economy stores.HouseholdCalculationServiceturns that error intoHouseholdCalculationOutcomeUnknownErrorand starts no Modal call.{"code": "internal_error", "detail": "The service may have created the household job, but could not confirm it was written."}with status 500 and noRetry-After, and logshousehold_job_outcome_unknownat error level witherror_type. 503database_unavailablenow means no job was written.policyengine_api/libs/simulation_household_calculations.pyonmasteralready reads sim's 500 as "the job may exist", and sim's coded 503 as "no job of the submission will run".If the job was written, it stays
pendingwith no Modal call ID, and the stuck-run sweep fails it.Move to the SQLAlchemy ORM
policyengine-simulation-persistenceandpolicyengine-stage12-persistenceis a typed declarative class, with each column, type, default, constraint and index unchanged. Adds session factories andbounded_session, which keeps the per-transaction statement timeout.Sessionand name mapped classes.Stage12PersistenceStore,ReportRunStore,HouseholdReportRunStoreandBudgetWindowStorerun on sessions.engine.pyholds anEngine, and no public function there returns anEngineorConnection.Behaviour kept:
INSERT ... ON CONFLICT DO NOTHING, followed by a read when the insert did nothing.UPDATE ... RETURNINGstatements withsynchronize_session=False.One added statement in the v2 Modal runtime: each store operation first sends
SELECT true, which makes the session check out its connection. Only that step is retried, so a retry can't repeat a write. For 50 store operations (118 statements), the SQL recorded before and after the change was otherwise identical in text, parameter names and batching.Rule
sim.persistence.orm-onlyCode that reads or writes database rows declares tables only as ORM mapped classes and runs every statement through an ORM
Session. The rule isenforced, and any finding failsprojects/policyengine-simulation-entry/tests/test_orm_only.py.tests/orm_access_support.py) reads each source file and reports Core table declarations, reads of a Core table from a mapped class, mapper orMetaData,Tableannotations, engines and connections outside a library'sengine.py,.executeon anything not named as a session, raw SQL outside model schema details, andsqlmodelimports.uv.lockcontains SQLModel.rules/fixtures/sim.persistence.orm-only/.sim.stage12.persistence-typed-dml-onlyno longer says "Core", which would contradict this rule.docs/engineering/skills/database-access.mdlists what it doesn't detect.For other open PRs: after this merges, code that uses Core fails the rule. #1096 and #1099 are to be rebuilt on the ORM.
Deployment changes
sqlmodelis removed from thepolicyengine-fastapi, gateway and executor locks.sqlalchemyandgreenletalso leave thepolicyengine-fastapiand gateway locks, where nothing imports them. No other package version changes.STAGE12_DATABASE_URL_SECRET_NAMEbetaandprod, naming a Google Secret Manager secretstage12-validate-infrastructure.sh, which writes the secret to the file given to--database-url-file.SIMULATION_SERVICE_DATABASE_URL_SECRET_NAMEbetaandprod, naming a Google Secret Manager secretMODAL_WORKER_DATABASE_URLpolicyengine-modal-worker-databaseVerification
345d0951, including every project's test job, the simulation database tests and the comparison with API v1's schema.python -m scripts.rules guardpass.Review
Each part had adversarial review rounds, and every substantiated bug was fixed:
These later changes had no further review round: the fixes for the last-round findings on #1101 and #1104, small follow-up edits on #1106, #1107 and #1109, the final fixes to the deployment check's single-connection and time-limit changes, and the two CI fixes that update stale tests.
Not verified
policyengine-modal-worker-databasecould not be confirmed.🤖 Generated with Claude Code